Meaning and the Mental Lexicon

This paper presents a network model of the mental lexicon and its formation. Models of word meaning typically postulate a network of nodes wi th connection strengths, or distances, that reflect semantic similarity, but seldom explain how the network is formed or how it could be represented in the brain. The model presented here is an attempt to address these questions. The network organizes semantically similar words into clusters when exposed to sequentially presented text. Lexical co-occurrence information is calculated and used to create a hierarchical semantic representation. The output is similar to semantic networks first described by [Collins and Loftus, 1975], but is created automatically. 1 I n t r o d u c t i o n The mental lexicon refers to the representations that allow word recognition on the basis of auditory and visual st imul i . The lexicon is understood as two linked levels of representation: The first level consists of formbased representations that reflect a word's phonological or graphemie properties. The second level contains semantic representations that reflect its meaning relations with other words [Marslen-Wilson, 1989], Pr iming studies are an important source of evidence for the semantic organization of the lexicon. When subjects are presented briefly wi th a letter string, followed by another, and asked to decide whether the latter is a real word, the response t ime when both strings are related is reliably faster than when they are unrelated. Priming effects can be found using st imul i that are graphemically, morphologically, or semantically related ([Taft, 1991] for a review). "The author is supported by a Medical Research Council studentship. Substantial progress has been made modelling the form-based lexical representations in the light of graphemie or phonological similarity [Plaut et a/., 1994], but there is currently no principled measure of semantic similarity. Word meaning is much more difficult to quantify. The network described here is an attempt to address this problem. It is inspired by two relatively independent approaches to semantic representation from cognitive psychology and computational linguistics. After considering each approach I describe the network's implementation and present results. The next section describes the structure and development of semantic representations, and how the model relates to previous work. Finally 1 consider the model's psychological relevance with reference to developing categorizations and semantic priming. 2 L e x i c a l s e m a n t i c N e t w o r k s A highly influential theory of lexical-semantic representation from cognitive psychology is based on the semantic network. A semantic network consists of a set of nodes and connections of varying strengths, or lengths, between them [Collins and Loftus, 1975]. Each concept is assigned a node, and connection strengths reflect the amount of conceptual relevance each node has to its partner. The stronger, or shorter, connections represent a high level of similarity. Weaker, or longer, connections hold between less related nodes. In a lexical-semantic network (LSN), each node represents a word and the distance between nodes reflects the amount of semantic similarity between each word. The Logogen model [Morton, 1979], Interactive-activation model and spreading activation accounts, are all types of LSN [Neely, 1991]. LSN accounts explain semantic pr iming effects in the following way: Each node has an activation level. When a stimulus is presented it activates all nodes in the network to some degree. If one node is activated strongly enough its activation wi l l pass a threshold and fire. The stimulus wi l l be recognized as that word. Each*time a 1092 NEURAL NETWORKS word is presented, activation spreads from the most activated node to nearby nodes, decaying over time. For example, i f 'doc tor ' is presented shortly before 'nurse', the node associated wi th 'nurse' wi l l reach threshold faster and fire sooner. Its resting activation level is raised by activation spreading from 'doctor' during the interstimulus interval. 3 Dataintensive Semantics Recent work in computational linguistics suggests that large amounts of semantic information can be extracted automatically from large text corpora on the basis of lexical co-occurrence information [Lund et al., 1995; Schiitze, 1993]. This approach is particularly well suited to neural network implementation [Finch, 1993] because co-occurrence statistics track conditional probabilities, and neural networks have straightforward interpretations as statistical models [Bishop, 1995]. The data-intensive approach to semantics is consistent wi th , and inspired by theories of meaning that emphasize the importance of use [Wittgenstein, 1958] (see also [Church and Mercer, 1993]). Lexical co-occurrence information reflects a word's distributional profile, which is a reflection of its use. The success of the data-intensive semantics research shows that, wi th a large enough sample, there is sufficient information in a strictly linguistic environment to recover much semantic structure. It seems plausible, therefore, to investigate the possibility that the brain makes use of such information. The recent discovery that semantic and associative priming effects in the lexical decision task are significantly correlated with co-occurrence statistics [Lund et al, 1995; Spence and Owens, 1990] support this possibility. Lund et al. constructed a highdimensional space on the basis of lexical co-occurrence counts. Words that were close together in the space gave larger pr iming effects than those further away. 4 Mode l l i ng the Lexicon LSN theories provide an intuit ive way to understand word meaning and its relation to priming. However, there is no theory of how the nodes of a network are formed, or how the distance (or strength) relations between them become organized. The data-intensive approach to semantics is an effective predictor of semantic priming, and reflects an influential approach to understanding word meaning. However, the approach requires an extremely highdimensional co-occurrence space for lexical-semantic representation. It is not obvious how such a space could be represented in the brain. The model presented below is a first attempt at explaining how the semantic level of the lexicon could be organized, consistent with the LSN and data-intensive semantics approaches, in a way that is computationally tractable and biologically reasonable. 4 . 1 O v e r v i e w o f t h e M o d e l The model consists of an input layer that picks out words from a text stream, a dynamic proto-lexicon which records co-occurrences between the present target word and words either side of i t , and a self-organizing map. The proto-lexicon is init ial ly empty and the selforganizing map weights are set to random values. 4 .2 I m p l e m e n t a t i o n P ro to l ex i con The proto-lexicon represents each word in terms of the number of times it has been seen to co-occur directly before and after each other word in the vocabulary. Specifically, in an n—word vocabulary each word Wi is associated with the vector normalized to unit length, where denotes the frequency with which Wj has preceded wi, before t, and denotes the frequency with which Wk has succeeded wi,. Thus at each time step, x represents the model's best guess for the conditional probabilities